{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting started with Ollama for Python\n",
    "\n",
    "This notebook provides a simple example of using Ollama with the Ollama Python library. To run this notebook, you must first set Ollama and the Python environment.\n",
    "\n",
    "## Setting Up Ollama\n",
    "\n",
    "To run this notebook, you will first install Ollama:\n",
    "Go to the [Download tab](https://ollama.ai/download) on the [Ollama website](https://ollama.ai/), select your OS, and follow the instructions. \n",
    "Note: Currently, there is support for MacOS and Linux OS. Windows support, according to the llama's website, is coming soon. \n",
    "\n",
    "Next, from the terminal:\n",
    "- Start Ollama - Once installed, use the `ollama serve` command to launch the Ollama server. You should expect the following output:\n",
    "\n",
    "```shell\n",
    "2024/01/28 07:34:07 images.go:857: INFO total blobs: 0\n",
    "2024/01/28 07:34:07 images.go:864: INFO total unused blobs removed: 0\n",
    "2024/01/28 07:34:07 routes.go:950: INFO Listening on 127.0.0.1:11434 (version 0.1.22)\n",
    "2024/01/28 07:34:07 payload_common.go:106: INFO Extracting dynamic libraries...\n",
    "2024/01/28 07:34:10 payload_common.go:145: INFO Dynamic LLM libraries [cuda_v11 cpu]\n",
    "2024/01/28 07:34:10 gpu.go:94: INFO Detecting GPU type\n",
    "2024/01/28 07:34:10 gpu.go:236: INFO Searching for GPU management library libnvidia-ml.so\n",
    "2024/01/28 07:34:10 gpu.go:282: INFO Discovered GPU libraries: []\n",
    "2024/01/28 07:34:10 gpu.go:236: INFO Searching for GPU management library librocm_smi64.so\n",
    "2024/01/28 07:34:10 gpu.go:282: INFO Discovered GPU libraries: []\n",
    "2024/01/28 07:34:10 cpu_common.go:18: INFO CPU does not have vector extensions\n",
    "2024/01/28 07:34:10 routes.go:973: INFO no GPU detected\n",
    "[GIN] 2024/01/28 - 14:13:47 | 200 |      238.75µs |       127.0.0.1 | HEAD     \"/\"\n",
    "[GIN] 2024/01/28 - 14:13:47 | 200 |     834.333µs |       127.0.0.1 | GET      \"/api/tags\"\n",
    "[GIN] 2024/01/28 - 14:13:59 | 200 |     229.292µs |       127.0.0.1 | HEAD     \"/\"\n",
    "2024/01/28 14:14:02 download.go:123: INFO downloading e8a35b5937a5 in 42 100 MB part(s)\n",
    "\n",
    "```\n",
    "\n",
    "**Note:** When running Ollama inside a Docker on a Mac machine it does not support GPU\n",
    "\n",
    "- Download a model - Use the `pull`` command to pull a model from the Ollama model registry. For example, the below code will download mistral:\n",
    "\n",
    "```shell\n",
    "ollama pull mistral\n",
    "\n",
    "pulling manifest \n",
    "pulling e8a35b5937a5... 100% ▕███████████████████████████████▏ 4.1 GB                         \n",
    "pulling 43070e2d4e53... 100% ▕███████████████████████████████▏  11 KB                         \n",
    "pulling e6836092461f... 100% ▕███████████████████████████████▏   42 B                         \n",
    "pulling ed11eda7790d... 100% ▕███████████████████████████████▏   30 B                         \n",
    "pulling f9b1e3196ecf... 100% ▕███████████████████████████████▏  483 B                         \n",
    "verifying sha256 digest \n",
    "writing manifest \n",
    "removing any unused layers \n",
    "success \n",
    "```\n",
    "\n",
    "Ollama currently supports a variety of LLMs, a full list is available here:\n",
    "https://ollama.ai/library\n",
    "\n",
    "\n",
    "To view the list of models on your local machine use the `list` command: \n",
    "```shell \n",
    "ollama list\n",
    "\n",
    "NAME            ID              SIZE    MODIFIED           \n",
    "llama2:latest   78e26419b446    3.8 GB  3 minutes ago     \n",
    "mistral:latest  61e88e884507    4.1 GB  8 minutes ago     \n",
    "vicuna:latest   370739dc897b    3.8 GB  About a minute ago\n",
    "```\n",
    "\n",
    "We will use the above three models in this demo.\n",
    "\n",
    "## Setting Up Python Environment\n",
    "\n",
    "There are many ways to set up a Python environment. I ran this demo inside a Dockerized environment, and you can run it locally using a virtual environment (which might be more straightforward). \n",
    "\n",
    "### Running Locally\n",
    "\n",
    "Setting up a local Python environment if running locally with venv is straightforward. Starting with setting up a virtual environment named ollama (you can choose any other name):\n",
    "``` shell\n",
    "python3 -m venv ollama\n",
    "```\n",
    "\n",
    "This will create a new folder named ollama under the root folder, which will include the following folders:\n",
    "```shell\n",
    ".\n",
    "├── ollama\n",
    "    ├── bin\n",
    "    ├── include\n",
    "    └── lib\n",
    "```\n",
    "\n",
    "To activate the environment, use:\n",
    "```shell\n",
    "source ollama/bin/activate\n",
    "```\n",
    "\n",
    "Next, we will install the ollama library using pip:\n",
    "```shell\n",
    "pip install ollama\n",
    "```\n",
    "To confirm that the installation was successful, open Python and test if you can import the library:\n",
    "```python\n",
    "Python 3.10.10 (v3.10.10:aad5f6a891, Feb  7 2023, 08:47:40) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin\n",
    "Type \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n",
    ">>> import ollama\n",
    ">>>\n",
    "```\n",
    "\n",
    "If you want to run the code on Jupyter notebook, open Jupyter and set the notebook kernel to the `ollama` virtual environment (or any other name you used to set the environment). \n",
    "\n",
    "### Running Ollama with Docker\n",
    "\n",
    "This repo has Docker setting for running Ollama inside a dockerized environment using VScode's Dev Containers extension. The caveat is that it is working but is still experimental, and more optimization can be done. The `.devcontainer` folder contains the Docker settings for this POC:\n",
    "```shell\n",
    ".\n",
    "├── .devcontainer\n",
    "   ├── Dockerfile\n",
    "   ├── devcontainer.env\n",
    "   ├── devcontainer.json\n",
    "   ├── install_dependencies.sh\n",
    "   ├── install_quarto.sh\n",
    "   ├── install_requirements.sh\n",
    "   └── requirements.txt\n",
    "```\n",
    "\n",
    "I created this repo using this [template](https://github.com/RamiKrispin/vscode-python-template) and modified the `requirements.txt`, `devcontainer.json`, and `Dockerfile` files.\n",
    "\n",
    "The Dockerfile set the Python environment, VScode extensions (e.g., Juypyter, etc.) and installed Ollama's Linux version. Once the folder opens inside the Dev Containers, the post-creat command is set to launch the Ollama server. \n",
    "\n",
    "The next step is to pull the required models from the terminal as described above using the pull command:\n",
    "```shell\n",
    "ollama pull mistral\n",
    "ollama pull llama2\n",
    "ollama pull vicuna\n",
    "```\n",
    "\n",
    "In the last step, open the notebook and choose the kernel using the ollama Python environment (in line with the name set on the devcontainer.json file).\n",
    "\n",
    "Alternatively, a more robust way is to use the official [Ollama docker image](https://hub.docker.com/r/ollama/ollama).\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ollama Python Demo\n",
    "\n",
    "Before running this demo, from the terminal start Ollama server using `ollama serve`, and make sure you pull the below three models - `mistral`, `llama2`, and `vicuna`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import ollama"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " The color of the sky appears blue due to a natural phenomenon called Rayleigh scattering. As sunlight reaches Earth's atmosphere, it interacts with different gases and particles present in the air, including nitrogen and oxygen molecules. Blue light has a shorter wavelength and gets scattered more easily than other colors, resulting in the scattering of blue light in all directions. Consequently, when we look up at the sky, we predominantly see the blue portion of the spectrum.\n"
     ]
    }
   ],
   "source": [
    "response = ollama.chat(model='mistral', messages=[\n",
    "  {\n",
    "    'role': 'user',\n",
    "    'content': 'Why is the sky blue?',\n",
    "  },\n",
    "])\n",
    "print(response['message']['content'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "The sky appears blue because of a phenomenon called Rayleigh scattering. When sunlight enters Earth's atmosphere, it encounters tiny molecules of gases such as nitrogen and oxygen. These molecules scatter the light in all directions, but they scatter shorter (blue) wavelengths more than longer (red) wavelengths. This is known as Rayleigh scattering.\n",
      "\n",
      "As a result of this scattering, the blue light is dispersed throughout the atmosphere, giving the sky its blue appearance. The deeper into the atmosphere you look, the more blue the light appears, because the shorter wavelengths are scattered in all directions and cannot be detected from far away.\n",
      "\n",
      "The reason why the sky appears blue rather than any other color is due to the specific wavelengths of light that are scattered. The human eye is most sensitive to the blue and yellow parts of the visible spectrum, so we perceive the sky as blue. If there were no scattering of light in the atmosphere, the sky would appear gray or black, as the only light that reaches our eyes would be the direct sunlight, which is mainly composed of red and orange wavelengths.\n",
      "\n",
      "It's worth noting that the color of the sky can vary depending on the time of day, weather conditions, and the presence of dust and pollutants in the atmosphere. For example, during sunrise and sunset, the sky can take on hues of red, orange, and pink due to the angle of the sunlight and the scattering of light by particles in the atmosphere.\n"
     ]
    }
   ],
   "source": [
    "response = ollama.chat(model='llama2', messages=[\n",
    "  {\n",
    "    'role': 'user',\n",
    "    'content': 'Why is the sky blue?',\n",
    "  },\n",
    "])\n",
    "print(response['message']['content'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The sky appears blue because the Earth's atmosphere scatters sunlight in all directions and blue light is scattered more than other colors because it travels as shorter, smaller waves. This is why the sky looks blue during the daytime. The sun appears to be shining down from the zenith (straight overhead) when the Earth is at its maximum tilt away from the sun, which happens in late summer in the Northern Hemisphere and early spring in the Southern Hemisphere. This is why the sky is often described as being \"clear\" or having a \"clear blue\" color during these times of year.\n"
     ]
    }
   ],
   "source": [
    "response = ollama.chat(model='vicuna', messages=[\n",
    "  {\n",
    "    'role': 'user',\n",
    "    'content': 'Why is the sky blue?',\n",
    "  },\n",
    "])\n",
    "print(response['message']['content'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Resources\n",
    "\n",
    "- Ollama website - https://ollama.ai/\n",
    "- Ollama source code - https://github.com/ollama\n",
    "- Ollama Python and JS release notes - https://ollama.ai/blog/python-javascript-libraries\n",
    "- Mistral LLM - https://huggingface.co/mistralai/Mistral-7B-v0.1\n",
    "- Llama2 LLM - https://huggingface.co/meta-llama\n",
    "- Vicuna LLM - https://huggingface.co/lmsys\n",
    "- A Dockerized Python Development Environment Template - https://github.com/RamiKrispin/vscode-python-template\n",
    "- Setting a Dockerized Python environment - https://medium.com/p/de2400c4812b"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "PROJECT_A",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
